posted 12-12-2007 06:36 PM            

Female was questioned today on a theft and I asked the detective if he interrogated her and he said, I said things like "Now is the time if you did something", and little hints-- but he never actually accused her and he said he never told her that there was "No doubt she did it." He said he didn't directly accuse her because he still wasn't sure if she did it. I asked if he "verbally beat her up" at all and he said "no."

I tested her and she barely failed one spot score (-5) on Re that missing money: Did you take it? on a multi issue test. I was very comfortable on the comparisons again and after the test she really came up truthful during interrogation. It seemed like a good apple and I am not easily fooled.

posted 12-12-2007 07:54 PM            

R6 Did you knowingly give that female any of that money?

R9 Regarding that money did you take it?

R12 Do you know that females identity?

I know they are a little odd, but so is the situation.

posted 12-12-2007 08:07 PM            

Before I get to your question though, there's not much difference, if any, between a "Did you do it...Did you do it Friday?" and "Do you know for sure who did it?" test. Some would argue multi-issue, but there's no great penalty.

In any event, your tests sounds like a screening exam. You can always break out the problem question and run it in a single-issue test and see what happens. Then you'd have your answer.

The reason we run one test like that is to avoid running three single-issue tests - to save time, but it seems to come at a cost of more time to figure out what went wrong - if something did go wrong. She could be lying. Did you interrogate?

posted 12-12-2007 08:37 PM            

I told her is she is telling the truth se will do fine on the test. I also told her I don't care what anyone told her but if she tells the truth she will be fine.

The case is this:

She (a Target cashier) tells a supervisor that someone just asked for change after buying a bag of Hershey kisses and she had the alarm go off that something wasn't right. She immediately notified her supervisor-right away. At the end of the night her register was $1,700 short. She has been employed there a year and never had a problem before. No CHIS.

She says that she was scammed and was rushed through the change process. This is tough to explain but I had cases somewhat similar and the victim was honest.

We have pictures of the suspect that asked for change. The whole incident is on camera right above the register, but (I was told by the patrol guy) that you can't see the numbers on the bills and you can't see how much she is giving out.

Obviously they want to know if she planned with this female to give out this money or was she tricked. I threw in R9 in case she took the money at another time and used the female as a cover. She had a decent hit on R9? -5 by hand -12.5 on Nates software.

I interrogated her and after I did another detective did as well. She went for nothing, after interrogation I kinda beleive her. Just by some of her quotes and non verbals. 20 y/o soft spoken female- my god it should have been a 5 minute interrogation.

During interrogation she said she knew she was giving out way too much money, but the girl was rushing her and she trusted her. I thought this was a beginning to a confession, but no.

She said she was thinking about how she stole 10 dollars from her grandmother during the test and the time she ate a pretzel without paying at work.

I get what you are saying on the multi to avoid three single issue tests. In a perfect world if there is a significant reaction we can sit down, have a talk, and resolve it through post test discussion.

I had a talk with the Sgt but the guys still like to have a talk with the suspect first, I hope they understand the whole interrogation issue, I exaplained it pretty well. These guys are new to poly.

posted 12-12-2007 08:58 PM            

quote:

---

During interrogation she said she knew she was giving out way too much money, but the girl was rushing her and she trusted her.

---

Could that mean she felt responsible for the theft? What did you pretest R9 to mean? It sounds pretty straightforward to me, but if you pretested it to mean she could have been an accomplice (she took an active role in the money going missing), she might have felt like she was because she knew something wasn't right, but she still gave the money away.

quote:

---

She said she was thinking about how she stole 10 dollars from her grandmother during the test and the time she ate a pretzel without paying at work.

---

Sounds like she was concerned with the CQs.

Would she come back for a test on that single-issue?

posted 12-12-2007 10:49 PM            

I think it is important in criminal specifics that the examinee knows someone thinks they are guilty. During the pretest I ask the subject about their conversation with the investigator. This allows me to form an opinion as to whether or not the conversation was enough to call it an "interogation" and postpone the examination. Remember the intensity issue lies mostly in the perception of the subject.

It also lets me pick up on the propensity of some investigators to tell the subject that they just want a polgraph examination to eliminate them as a suspect.

If the investigator has done that I tell the subject that I don't run examinations just to clear people. I tell them that the reason that they have been asked to submit to polygraph is because someone believes they are guilty and I have been asked to resolve that belief, one way or the other.

Polygraph aside, $1700.00 would have to be close to a world record hit for a quick change artist on a single register. Considering that a $100.00 bill is the largest denomination in common circulation, the scammer would have to turn it over 17 times in a single transaction. Most pros will turn their original bill over 7 or 8 times. There is something very wrong here; either your clerk is dirty or there os some other explanation for the high loss. It also may be part quick change scam and part embezzlement.

If I got another chance at polygraph I might want to go back with a multi facet,
Did you take any of that money?
Did you plan with anyone to take any of that money?
Do you know the name of the person who took any of that money.
The followup with a single issue if needed.

Considering the improbability of a $1700.00 quick change I think you should feel comfortable to interrogate with confidence. Even if you have a low spot score.

Butthen that's just one man's opinion. Happy Hunting!

------------------
Ex scientia veritas

posted 12-13-2007 12:04 AM            

This is a matter for which diagnostic accuracy should remain a priority.

More in a bit.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 12:08 AM            

Buster, if you can email me the test data, I'll put it on a page with a secured link so people can see the data. You can even fax the charts to me at 720-554-7677 and I can make a secured link from that.

-----------------

Here's a very real mathematical problem which I've been working on, and which pertains to this test.

First, when reviewing charts and score, its very difficult and perhaps sometimes impossible to compare one examiner's scores to another. In blunt terms its not hard to do, but mathematically and empirically (scientifically) it is meaningingless to do so. I say this for two reasons:

• examiners working in different programs, from different schools or trainging, and with different administrative and QC protocols sometimes use different scoring systems (i.e., reaction features, ratios, etc.), and
• even when we use similar scoring systems we describe our results in terms of point totals that have no known mathematical significance or p-value.

That is a hint. We could easily compare one examiner's scores to another if we discussed our results in terms of mathematical/statistical p-values and not some arcane point total for which we may or may not adequately understand the statistical significance, distributions, standard error ranges, confidence intervals, or interrater reliability. We could of course do a small study to determine those things, and then we would know the corresponding p-values for any point total for any different scoring system.

That would help us begin to silence (or at least math-slap and ridicule) the likes of Sergeant1107 who rants at anti about Daubert requirements pertaining to known rates or methods for estimating the likelihood of an erroneous test result.

It is both simplistic and erroneous to translate frequency type statistics, like a median/mean/average accuracy of 86%, as reported in the NRC/NAS report (2003), as a form of description for the significance or probability of accuracy (or error, if you calculate the inverse) of a single test. Its wrong.

Much as I like the idea of Marin certification, there is an inherent fallacy at play in the assumptions of the protocol. Its the same one that novice psychological evaluators get into, when they write something like "individuals with similar scores...," or "individuals who respond similarly...", or "people with similar test results..." (whatever). The point it, we don't really care about similar people - we are concerned about THIS GUY - what is the likelihood that this test is correct or incorrect. 20 years of experience, 12 years of college, 5 advanced degrees, who care. The greatest polygraph examiner in the world may have had a bad day. Similarly, the guy right out of polygraph school may have conducted a stellar examination. Whether someone scored the Marin certification data accurately enough doesn't really tell us much about whether the exam in question is adequate or not. For that we need some form of quality assurance protocol for evaluating a single examination.

OK, enough of that mini-rant.

Back to the point about this test.

Multi-facet and mixed-issues tests are great because they allow us to increase or broaden the test's sensitivity to deception about a range of concerns.

Multi-facet and mixed-issues tests also stink because the really complicate the parsing of test results among the several distinct questions.

There is some evidence, going back to Barland, Honts, and Barger (1989) and perhaps even earlier, and more recent, that mixed issues tests may have lower sensitivity levels than we might guess or assume. More on that in a bit. There is also evidence that while multi-facet/mixed-issues exams may indicate when people are lying, the evidence is weaker regarding whether they can actually tell us what question someone is lying to. I know our individual anecdotal experience tell us otherwise, but we are foolish to ignore data if we want to prevail in arguments against articulate foes. One way of thinking about this is that people fail (are deceptive to) the examination, not the questions - perhaps especially in screening situations.

So, why might this be?

First, a metaphor. A single issue test (like a good 'ole three question ZCT) is essentially like shaking one tree to determine the presence of apples. Apples would be determined by something falling from the tree, and we get to shake that tree nine (9) times to be sure about our conclusion whether there are or are not apples in the tree. In psychological terms we are looking for a significant reaction to a stimulus (test question), and when we present the stimulus repeatedly, and find a statistically significant absence of reactions, compared to the subject's reactions to other/comparison simulus, then we consider it reasonable to conclude there is also no cause for reaction - cause for reaction is assumed to be some form of involvement in the allegation, issue, or event under investigation. What is important to polygraph as a science is that we must move towards a statistically valid approach to our assertions about the presence or absence of those "significant reactions" Significant means "statistically significant." Without statistics to back up our assertions, then accusations of pseudoscience are very difficult to refute, because conclusions are impressionistic, anecdotal, unreliable, and therefore un-replicatable.

The real problem: we don't know the distributions of spot scores. What that means is we really don't know what the means and standard deviations are, and we can't calculate a level of significance or p-value. Neither can we calculate a standard error or confidence interval for spot scores. Presently, only OSS-3 can do that, because OSS-3 standardizes spot scores to the same distribution as total scores, and uses the distribution for total scores as an approximation for spots. We don't really know if or how other algorithms have address this mathematical problem for calculating the significance of spot scores. I doubt any examiner would want to go to the hassle of calculating the significance of hand-scored spot totals.

A lot of people understand the challenges inherent to studying spot scores: access to enough confirmed data so that we can study the distributions of confirmed spot by spot results. Most do not understand the modern simple solution: monte-carlo experiments, using computer simulations built of massive strings of random numbers (like 1000 distributions of 100 samples of 100 simulated exams, with three RQs and three CQs each, composed of four components = about 240 million random numbers). With a monte-carlo we could know the confirmation/criterion of every spot. It can be done, using software that is available for free. Our detractors know about monte-carlo simulations and are using them in arguments against us. We might as well learn to use them to our advantage - look around and you'll be amazed to learn how many things are modelled with monte-carlos these days, traffic, buildings, cities, medications, battles, video games, stock and financial markets, sales and manufacturing forecasts - anything any everything can be simulated and studied.

There is hope on a smaller and more human scale too, and it would not be difficult to complete a simple study that would provide data and answers to these concerns. It would require participation from people training, endorsing, and utilizing the different scoring systems. Once done, we could compare the results from different examiners, using different scoring systems, in statistically responsible and meaningful ways.

Now back to the metaphor. If we stimulate/question the issue or shake the tree rigorously (3 RQs x 3 charts = 9 times) and to our satisfaction and then nothing falls out, then we are reasonable in concluding there is nothing in the tree (no involvement).

With a multi-facet or mixed-issue exam like a USAF-MGQT exam with three RQs, we have three trees to shake, not one, and we get to shake each tree or stimulate each issue three times. The volume of stimulation/shaking applied to each issue/tree is substantially reduced. This may be related to sensitivity concerns. This is primarily a concern with mixed-issues exams - for which the issues are conceptually distinct (i.e., it is conceavable that a person could be involved in one or more target issues while being completely un-involved in other target issues).

To carry the metaphor further, our present uncertainty about the questions themselves is like having to shake the trees while blindfolded. If we notice, after shaking each of the three trees three times, that something has fallen out of the trees and onto the ground, we cannot be quite sure which tree the thing came from - until we interrogate the subject and conduct further testing of the remaining issues.

That is the sensitivity problem.

Now the specificity problem.

Mult-facet and mixed issues exams are subject to well-known mathematical priciples - one of which is called the "addition rule," which pertains to dependent probability events. Multi-facet and mixed-issues polygraphs are dependent probability events because passing any question requires also passing the other questions - the questions are called dependent because they affect the results of the other questions. You cannot pass one question while failing another (remember the data do not yet support the idea that these exams can reliably tell us which question an examinee is lying to).

So, if we did know the significance or alpha level represent by our point totals, and if we set our alpha level (decision threshold or cutscore) at .05 (which is a common threshold for type-I error tolerance), that would be we are willing in advance to tolerate a 1 in 20 chance that our results are incorrect. But that alpha pertains to each distinct question. So we have to add 5 percent for the first RQ, + 5 percent for the second RQ, + 5 percent for the third RQ, and so on. This phenomena is called "inflation of the alpha level," and is a phenomena known to every statistics professor in the country - the result of which is an increased likelihood of false-positive results. (this is a take-home point people). Its why laboratory researchers have deprecated the practice of conducting multiple simultaneous significance tests - there is an increased likelihood of false positive results. Its also why researchers favor omnibus statistics like analysis of variance (ANOVA) methods when faced with the need to evaluate multiple simultaneous significance experiments. Its also why OSS-3 uses a non-parametric ANOVA algorithm for decision parsing of mixed-issues exams.

In hand-scoring, it is unlikely to get field examiners to use ANOVA methods. There are corrected statistical models for calculating multiple simultaneous significance tests. Tukey's is one. Bonferonni is another. Bonferonni has the advantage of being simple - we apply a "Bonferonni correction," and divide our specified decision alpha of .05 by the number of distinct questions. So, our alpha of .05 become .05/3 = .0167, which we call a "corrected alpha" or "Bonferonni corrected alpha." That corrects the inflated alpha and protects against false positives.

Inconclusive rates are also subject to the effects of the addition-rule. Because any inconclusive spot, multi-facet or mixed-issue exam forces and inconclusive overall result - the questions are dependent for inconclusive results. We therefore add the likelihood of an inconclusive results for each distinct question. The more questions, the more likely the test comes out inconclusive. If we assume that 5 to 10 percent of single issue exams may be inconclusive, we may be correct in assuming a noticeable increase from that with multi-facet or mixed-issues exams. That is, unless we examiners take other steps to reduce inconclusives (i.e, more charts), the principles remain the same.

The real problem is that in our present hand-scoring systems we don't even know the significance of our total scores, let alone the spot scores. We are told, with no supportive data, to use +/- 6 for totals and +/- 3 for spots, without any explanation of the distributions or significance of those values. We have allowed ourselves to be naively satisfied with simple descriptives of accuracy outcomes with a few sample datasets.

Consider this: if we assume that a total score of -5 corresponds to a p-value of <.05 we still don't know whether a spot score of greater or lesser magnitude than -5 would correspond to alpha of .05 for spot scores (or .0167 if we are using a corrected alpha).

So, at present we cannot really calculate the likelihood that your test result is an error.

It would be simple to study this problem, and answer these questions. All it would require is a cohort of examiners using the various common scoring systems(i.e., Backster, Utah, Federal, Horizontal, etc) to score a sample dataset and send the data to some SOB who stays up too late thinking about these kinds of things.

Alright, enough math-ranting for one evening.

Niters.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 06:10 AM            

In looking at this incident there appear to be several scenarios that might explain the missing money,

A. No theft occurred and there is some other unknown error or act that explains the missing money in a way that renders both the Cashier and the Customer innocent.
B. The Cashier was victimized by one of the worlds all star quick change artists.
C. The Cashier and the “Customer” conspired to steal the money.
D. The Cashier and “customer conspired to steal some of the money and there is some other unknown error or act that explains the remainder of the missing money.
E. The Cashier and the “Customer” both stole money from the register as separate acts.
F. The Cashier stole all of the money and used an innocent “Customer” in an attempt to conceal her crime.
G. The Cashier stole some of the money and some other unknown error or act explains the remainder of the missing money.
H. The Customer stole some of the money and some other unknown error or act accounts for part of the missing money.

Scenario A cannot be resolved by a polygraph examination of the Cashier.

A DI or NDI single issue series “Did you steal or take (pick the ONE you were trained to use) any of that money?” would resolve B, E, F, G and H and possibly C and D, but might allow a psychological escape on C and D if the examinee could attach herself to the concept that she didn’t “actually steal any of the money” because she just handed it to a friend who then stole it.

A DI or NDI single issue series “Did you plan with anyone to steal any of that money?” would resolve C and D. I would argue that this question, used as the third relevant on a good old three question ZCT would turn it into a multi facet test because she could have planned the theft and still psychologically escape on the other two questions because she didn’t physically take any of the money.

But if you remove the words “with anyone” from “Did you plan with anyone to steal any of that money” changing it to “Did you plan to steal any of that money?” I think you probably remove any possible psychological escape and whether she planned the theft by herself or with the customer is irrelevant for the purpose of determining whether or not she is involved in the crime and only comes into play when you begin to investigate the level of her involvement in the crime.

I think you would have a pretty tight single issue test that would cover all scenarios but A with the following relevant questions.
Did you steal any of that money?
Did you steal any of that money from your cashier station?
Did you plan to steal any of that money?

(Feel free to use “any of that $1700.00 reported stolen on November 14th or whatever date, if you are concerned that her admission to a minor previous theft might cause her to react to the RQ. I generally put this information in the SR)

This points out to me why I should refrain from shooting from the hip or using the “Same old RQs and CQs” on question lists like I did last night. This is why I don’t like to have an investigator call me and say “I need a rush polygraph, I can be there in 10 minutes” That doesn’t give me proper time to think about the issues that need to be resolved and make sure that I’m asking the best questions for the situation.

But I still go back to my original response, a $1700.00 quick change, just doesn’t pass the smell test. ABSENT POLYGRAPH, I suspect your clerk is either involved or maybe the supervisor who counted out her register found a way to pocket some early Christmas money using the quick change to cover her theft.

------------------
Ex scientia veritas

posted 12-13-2007 08:17 AM            

I'll go one at a time.

Barry:

The detective said she said she would pay the money back because she felt responsible. Yes thats odd, but during the pretest I really tried to separate and explain her feeling responsible from her taking the money herself or planning with someone to take the money. I did R9 as did you take that missing money from the register. I then asked what that meant to her and she explained it back properly.

I didn't see the tape, neither did the detective...only the patrol guy did. I would assume if you studied the tape for a full 8 hours you could see if she was taking money here and there unless she is really good. I dont know the exact camera angle, blind spots etc.

I don't know about a second test. I can mention it. Its up to the detective handling the case. I don't know if she would agree, she was pretty upset after a lenghty interrogation.

Ebvan:

I really like you asking about the discussion with the detective. I really like the reasons and plan to add that immediately.

Yes that is an ungodly amount of money for a change scam. Its very strange. I mean we have been fooled, but this girl seemed like a real goody two shoes, those scammers usually give a vibe. Frustrating case. That unidentified females pic is going to all mall managers and maybe the detectives will come up with something. To be honest, I think we admit we are all a little arrogant, I wanted to resolve it yesterday.

Your (R)questions sound very similar to mine
-maybe you cleaned up the wording a little.

Nelson:

You need to open a polygraph school!!!! I have to read your post againto get the full benefit of all of the aspects. I will reply to it shortly.

[This message has been edited by Buster (edited 12-13-2007).]

posted 12-13-2007 08:48 AM            

well, maybe on the caffeine, but only in social situations,

and a little bit of math won't hurt,

I can control it,

no-one will know if I just sneak in a little bit of math into the discussion...

---------------------------

I like "take" better than "steal." "Steal" is a legal term, "take" is what people tell themselves and others. Remember, delinquents don't steal cars they borrow them, with no intention of keeping them for themselves. Also, nobody uses illegal drugs - they "experiment."

I believe this is the second most common form of lying among adolescents and adult - called re-labelling. Best re-labelling lie of all time: a former president who said he did not have sexual relations with an intern.

The number one most common form of lying is, in my opinion, telling partial truths or half-truths - attempting to leave out bits and pieces that make a difference. Best partial truth of all time: a former president who says he didn't inhale.

alrigh, enough of that foolishness.

-----------

These are good questions to me

quote:

---

Did you steal any of that money?
Did you steal any of that money from your cashier station?
Did you plan to steal any of that

(Feel free to use “any of that $1700.00 reported stolen on November 14th or whatever date, if you are concerned that her admission to a minor previous theft might cause her to react to the RQ. I generally put this information in the SR)

money?

---

I might adjust "steal" to "take" and I might change "your cashier station" to "the cashier station."

But I believe they would work just fine as they are.

We will all have differing opinions regarding question language. We need differing opinion, else we're all just head-bobbering minions who would be easily misled by the guy who does have an opinion, and we'd never think for ourselves. Like unseemly parts of the anatomy people without an opinion are full of S$%^t.

What is important, is whether we are confident the questions would work as intended, even if we would do it different (I think this is actually language or idea hi-jacked from Jim Wygant - but it powerful and on-point).

--------------

I also like the method of delimiting the issues, drafting the questions and evaluating which of the issues the questions seem to capture.

--------------- I like these lines this morning only for visual variety, because I'm tired of typing long paragraphs

-----------------

Another thing to be aware of is that we polygraph examiners allow ourselves to "psychologize" way more than any psychologist that I know - when we start using concepts like "psychological escape." Human experience, for non-psychotic persons is robust enough that I doubt someone could consciously compartmentalize their thoughts/memories/experience/conditione-response-potential, as effectively as we sometimes suggest.

Modern psychologists, at least in the US, don't often allow themselves to accept a psychological explanation that is unproven by data. That is partly why they think we polygraph examiners are silly - we make all kinds of psychologized explanations that are unsuported and inconsistent with a simple parsimonious approach. It is important that our question stimulus and language be logically focused and adhere to the correct prinicples, but its probably good to avoid excessive psychologizing about how people react internally to certain details of the questions. Its the stimulus as a whole to which they react.

With that in mind, I wouldn't be all that concerned about scoring a ZCT as a single issue exam even if R10 included "play with anyone." Those words for me, do not include a dimension of any different behavior other than stealing or taking the money.

.02

r

--

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 08:59 AM            

When I find I have written one of these questions I try to find another way to ask the same information.

Does anyone else think this is worth worrying about?

------------------
Ex scientia veritas

posted 12-13-2007 09:08 AM            

I do not have e-mail setup. I will be faxing the charts today. In case you are not familiar with my technique- its a little different. It goes like this:

Chart 1 - Demo
Chart 2- Green/Red (Silent Answer)
Chart 3- Green/Red (Out Loud)
Chart 4- Red/Green (Out Loud)

posted 12-13-2007 10:02 AM            

After all this brain-damage I'm looking forward to seeing some charts.

I'll start another thread on the mult-facet thing - its important.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 10:25 AM            

quote:

---

Another thing I think we should address as examiners is the "Rhythm" of our RQs. If an RQ has a convoluted rhythm when spoken or contains a parenthetical phrase that makes it sound "odd" when spoken, I worry that the structure might cause an orienting response to the "rhythm" rather than the substance of the question.

When I find I have written one of these questions I try to find another way to ask the same information.

Does anyone else think this is worth worrying about?

---

Another good point. I made a mental note about that last night, but didn't get to it in all the excitement.

What you're describing is called "novelty," and producing and "orienting reaction," which is distinct from the "defensive reactions," in which we are interested, according to theory. The problem is that human psychology and human physiology is sloppy stuff and not all that tidy. It is unlikely that our observed reactions are completely either/or (i.e., OR or DR). It is more likely that these two are not completely separated but mixed up a little bit, wherein people are are truthful might experience defensive reactions. For example: pull anyone off the street and ask them about sexually assaulting their own children - include their kids names in the test questions. Do you think you'll see some form of reaction to the RQs? I do. I'd be surprised to see nothing. Its also possible that people who are guilty will experience and exhibit novelty or orienting reactions in addition to DRs - such as when the questions are worded oddly, or excessively complex.

I am convinced that EDA reactions, have less to do sweat than they do acetylcholine. I've been using a great EDA from limestone that works well with everyone except those taking boatloads of meds. It works with concrete workers and brick-layers - who's skin is dried out and leathery and whose eccrine sweat pores are quite damaged from whatever makes concrete and mortar work. All it takes is a wet-gel sticky electrode or some potassium chloride on the brass finger plates (my favorite - obtained from Axciton some years ago).

Acetylcholine is the enervating neurotransmitter in the sympathetic neurons in the skin, and is the same enervating neurotransmitter in the sympathetic neurons in the prefrontal cortex (the part of the brain responsible for attention, concentration, choice, judgement, complex problem solving, some aspects of personality, and all the mental gymnastics that go with lying or concealing information.)

Now guess why distraction, ADHD, singing songs, and outside stimulus affect the EDA. And guess why the test works with silent-answer and yes-answer series.

Our polygraph gear cannot distinguish between DRs and ORs. We have to conduct the exam and present the stimulus in a manner that we are confident that observed reactions are most likely DRs to the RQs stimulus - compared with DRs to the CQs.

If a question sounds odd to you, it probably sounds odd to the test subject - assuming no great cultural or sub-cultural misunderstanding.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 11:19 AM            

I got your fax, but only the PolyScore Rank results and 2 pages of Exam 1 Chart2 came through.

Can you borrow an email somewhere and send the test. I'm curious and I'd like to score the exam with OSS-3. It looks like a Lafayette exam, and would only take a couple of minutes. I'll even pay the kinko's cost -

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 11:29 AM            

I also tried to send you a VSA thing my Sgt got in the mail, I found it amusing.

If there is an easy way to e-mail this let me know, I tried it before and it didnt work, I was going to have my mentor set it up, but I forgot when I was there for an update class.

Sending again now.

The OSS I use is tricky for multi issue because it doesnt recognize our spot scores because we rotate relevants.

[This message has been edited by Buster (edited 12-13-2007).]

posted 12-13-2007 11:46 AM            

gimme a minute.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 11:58 AM            

A simple concept that might be under-considered is that to interrogate or repeatedly question before a polygraph, the relevant target issue does the obvious over-sensitization (Ray would rightly accuse me of over psychologizing here) of the target issues--BUT, we should also consider that repeated questioning before a polygraph test can HABITUATE the examinee to the target. So you should remind your chief that when he questions subjects, they become less and less sensetized to the target, making for a possible false negative.-----------------------------------------------------------------------------------------I'm with Ray, I love dashy lines, and I often use them on forums in the place of those oh so boring commas.

You can now resume the regularly scheduled heady discussion.

posted 12-13-2007 12:13 PM            

quote:

---

A simple concept that might be under-considered is that to interrogate or repeatedly question before a polygraph, the relevant target issue does the obvious over-sensitization (Ray would rightly accuse me of over psychologizing here) of the target issues--BUT, we should also consider that repeated questioning before a polygraph test can HABITUATE the examinee to the target. So you should remind your chief that when he questions subjects, they become less and less sensetized to the target, making for a possible false negative.

---

No. I wouldn't.

Both sensitization and habituation are well understood psychological constructs. I believe you are correct about both possibilities, and we don't really know the answer to questions about how the possibilities play out, when we might reliably anticipate their presence or influence, or how possible to mitigate or exploit them.

Its always OK to wonder about something.

Its only too much psychologizing when we begin to assume we know, understand, or anticipate the effects of a construct which we do not really understand.

Good points.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 12:31 PM            

40 subjects---20 programmed thieves, 20 innocents.
10 thieves and 10 innocents are subjected to a pre pretest of 50 target sensetive questions (a mock interrogation) before the pre-test begins.

posted 12-13-2007 12:47 PM            

phpBB is fully featured and free/open-source

its used everywhere.

he's already set up phpBB2,

however, version three (phpBB3) is just around the corner, and has

user logging - which looks very useful for things like hunting moles.

we are awaiting the release, a php upgradde on ralph's server, and for him to free up some of the time he spends pimpin' around with paid work.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 07:41 PM            

http://www.raymondnelson.us/qc/120712.html

user: polyguest
password: torquemada

Thanks Buster.

I haven't scored the test yes, I just cut a pasted the graphics.

Hey everyone - phpBB3 gold release was today.

That's one of the things we've been waiting for. Time to start heckling Ralph. Then we can put charts right into this forum. And why couldn't we modify one of those forum poll template to perform online scoring?

More to do.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-13-2007 09:13 PM            

I put the wrong link earler - fixed now, but here it is again.
http://www.raymondnelson.us/qc/120712.html

user: polyguest
password: torquemada

OSS-3 is built of the stuff most of us dissociate forget as graduate school trauma from the past (stats classes).

I don't expect everyone to be interested in this stuff, or care to take the time to understand it. But some might, and just the availability of it - and knowing about it makes us smarter, and less likely to be intellectually bullied by some dip-wad like digithead at anti (he's apparently vanished).

r

posted 12-13-2007 10:26 PM            

I don't recall exactly what the NRC's concern was with former methods.

I'm sure there are some smarter folks than I from DACA who read this forum, and they could probably do a better job than I explaining this stuff, or any of the ramblings of the last two days. I'm also sure that none of what I've provided is unfamiliar or revolutionary for any of them, as OSS-3 is really just the basic building blocks of statistics.

ROC stands for Receiver Operator Characteristic , and I couldn't tell you why they call it that - I dunno.

ROCs come from signal processing and signal detection theory, and where the practical invention of wartime radio operators who were faced with the challenge of detecting weak signals in a noisy field of radio freuquencies. That is not unlike the challenge of finding meaningful physiological response data among the myriad of things going on in polygraph tracings. Radio operators could select any of a range of frequencies and settings - they had to determine which ones worked best, and what noise to pay attention to (because it was good signal).

There is no such thing as a perfect test, and we've learned to manage the limitations of our methods by "optimizing" our tests around their limitations. That's why we use different decision rules for screening and diagnostic tests - they have different priorities, so we manage biases and limitations differently.

In polygraph, and many other testing contexts, we can adjust our accuracy in different way, and we can control for false positives, false, negatives, or inconclusives.

Jamie McCloughan and others have reminded us that we can probably never control for all of the types of errors at once. Reducing all at once is really a construct validity problem that we can't solve yet.

We can adjust our cutting scores (or more correctly, our alpha levels), to prioritize either sensitivity or specificity.

Sensitivity refers to the ability to notice the issue reliably (without missing important things). Specificity refers to the ability to restrict positive result to only those case that express the issue of interest (with minimal false positives). An ideal test would be catch every liar, reject ever truthteller, but their is no such thing as a perfect test - anywhere. We do the best we can. I do thinks that some of our historical methods can still be improved upon - in terms of understanding statistical significance, accounting for sensitivity and specificity, and interrater reliability.

Anyway, the only way to really understand how a test performs is to evaluate with the various cutting scores. The problem with that is that you end up with a table (lists) of different accuracy scores that describe the test's capabilities. There is nothing inherently wrong with that but its difficult to compare two or more different test or methods with each other, because you have to wrap your brain around a whole table of numbers just to understand one (each) test.

It would be preferable to have a single numerical index that describes the performance of each test - ideally a numerical index that captures information about both 1) a test's ability to notice the issue of concern, and 2) a test's ability to reject data or issues not of concern. In polygraph and other testing situations, we call those requirements sensitivity (to deception) and specificity (to truthfulness).

But wait, that's not all, because we also want our single numerical descriptor to capture and describe information about the performance of a test across a complete range of alpha levels (cutscores), and do so without a list or table of numbers - that way we can more easily compare one test or method to another.

So how do we achieve that tall order? ROC plots.

ROC plots or curves as simply graphs that compare the rate of true positives (sentisitivity level) to false positivies (1 minus the specificity level) across all possible cut scores (alphas).

If we draw the plot is a square matrix divided into 100 divisions along x and y axis, then we can easily plot percentile rankings for sensitivity and 1-specificity for all levels, then for each level we measure and sum the sensitivity level and divide that sum by the N. The result is what we cal the Area Under the Curve or "A" - sometimes called AUC or AOC (not sure why, and I could be wrong on this).

A is generally calculated with no inconclusive zone - each case is either correct or incorrect - for every decision level (alpha or cutscore).

The nice thing about A is that it captures in a single numerical index, all information about sensitivity, specificity and test performance across a complete range of decision levels.

Nice, huh?

But wait, if you call before mid-night we'll add even more.

ROC A statistics are a really important upgreydd over other statistical metrics, because they are resistant to base-rates, that is changes in base-rates do not affect the calculated estimation of A.

So, the NRC may have been concerned with former methods for that reason.

Anyway, A tells us about the optimum possible performance of a test across all conditions. It does not really tell us the accuracy percentage of a test - nor are we really all that interested in percent correct, because its not really all that informative to statisticians, who are usually more interested confidence intervals and resistant numbers that can be more informative when we generalize our new knowledge from one study sample to another or from study situations to live situations.

Hope this helps.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-14-2007 12:31 AM            

The ROC can help an examiner find the confidence level for the results of the examination they conduct in the field minus a known base rate, which we do not have in the field because subjects and their condition are random.

Basically, this could help do for polygraph what DNA has capitalized on in their field. That is, the results of the examination would answer a question with a measurable level of confidence in that result.

posted 12-14-2007 02:08 AM            

"Confidence" has a carefully understood meaning in statistics. I believe you mean p-value, which is our estimation that an observed result is an error (actually, a type-1 error). Alpha is our stated type-1 error tolerance level. Type-1 error is to reject the null hypothesis when it is true. the Null Hypothesis is that there is no significant difference (between our observed score and that of a truthful or deceptive person).

If you understand that as "confidence," that's OK, but confidence usually refers to "confidence interval" which recognizes at all results are estimates and may vary from day to day. The confidence interval is an estimation of how much they vary, and where we anticipate the normal range to be - in other words the range in which we expect the score to occur within on any given day, with some level of confidence attached to that.

So, just like our our estimates are biased, and we can estimate our bias, our confidence ranges can have confidence intervals of their own.

Cay you say - "brain damage."

I knew you could.

Its also not quite correct to say "minus a known base rate." Remember ROCs are resistant to base rate changes or differences. Its nice to know a base-rate, but it doesn't really matter what it is for an ROC to be informative.

OSS-3 is also built on resistant inferential statistical models, meaning that the p-value is the p-value, regardless of the base-rate.

I would have happily pursued a bayesian final, but Don Krapohl suggested retaining the inferential final like the one used in OSS2.

A few weeks about I built a monte-carlo spreadsheet to simulate ZCT exam. Among other tricks, it can show you that the inferential model retains balance of sensistivity and specificity even with very-low base-rate conditions (like 1:100 or 1:1000). I can send it to you if you want to see it.

I wouldn't hold my breath about ROCs doing for polygraph what DNA did elsewhere. I don't know anything about DNA, but I would guess that they observe much smaller variancy, and may deal with better signal to noise levels.

Monte-carlo models will teach us as much or more than ROCs.

I think the fears in polygraphy is that we'll never make sense out of the math with mixed issues and multi-facet exams, and that we'd never be able to validate the constructs. And the fear of After this last year, I'm over the one about the math. Construct issues are resolvable too, but we have to be willing to learn. Now, its just the crappy data that bugs me.

ROCs are just another statistical tool for studying and comparing data.

They are just plots of true-positives against false positives, for all decision thresholds.

Its not exactly the same as the likelihood of a correct or incorrect result for a single exam.

posted 12-14-2007 09:33 AM            

Thanx for posting my charts. They weren't the best looking ones I produced. I'm still leaning false positive, if not she did a hell of a job in interrogation.

posted 12-14-2007 10:10 AM            

I'll try again.
http://www.oss3.info/7-10-07_ROC_plots_for_componenents_1.1_to_1.pdf

The link above had three ROC plots: one for P (pneumo), E (EDA, and C (cardio).

First remember that ROC is concerned with sensitivity and specificity, which are the same as percent correct and incorrect, calcluted with inconclusives (Yep, deal with it. That's how its done.)

When we report sensitivity and specificity for a single sample (which is becoming less and less interesting to researchers who are familiar with bootstrapping, monte carlo techniques) we generally do that with a best-case-scenario - using the best cutpoints we can determine.

We know that observed decision accuracy will change when we change the cutpoints. (But we almost never have an adequate understanding of the statistical significance of those cutpoints.)

ROS plots are a visual graph of an ALL-CASE-SCENARIO. Imagine if you could graph the accuracy of results across all possible cutpoints. You can. Its call ROC.

Look at the plots again, and notice the diagonal line from the lower left to upper right. That is the chance line.

The data in the plot, from left to right, represent all possible decision points. thats "cutscore" to you and me, but researchers call it "alpha," because it is a mathematical description of our type-1 error rate. This is another point where we need to learn some scientific vocabulary so that we don't look silly or stupid to our adversaries. Say ".05" (pronounced "point-oh-five") to a researcher or scientist and they know what you mean. Say "-6," and they think you are a pseudoscientist engaginging in polybabble. They ask "what's -6" and we say "that's where he's deceptive,"

If test performance were linear, with a 1 to 1 corresponding change between true-positives and false positives (unlikely), we would see our data plot along that diagonal.

If the polygraph (or any method for making dichotomous classifications) were 50% accurate the data would plot along the chance, line. Mosts tests are not linear across various decision levels, nor is it desireable that they are. Most test will form a curve. ROC curves above the diagnoline line are good, and indicate better than chance test performance. ROC curves that are below the chance-line/diagonal are bad - worse than average test performance. (Actually its not that simple, because if you had a really poor ly performing test you could invert your procedures and bingo its good - the curve would be above the line. Its the chance line that's bad.

Now, look at the plots again. Find a Sharpie or Crayola and color in the part of each graph that is below and to the right of the curve. Hint: use someone else's computer or print the graphs before this part.

The part you just colored is call the Area Under the Curve (AUC) or just A. If you could measure the proportion of the graph that you just colored in the value would be a decimal between zero and 1, and provides a measurement of a test's peformance in one single numerical index - across all possible cutpoints.

A represents, in general terms, the likelihood that a test result is accurate if an examiner selected a case at random, from a population of unknown base-rate, and used a cutpoint selected at random. I know that probably doesn't sound like it helps much, but it does.

Because A is a single number - it is straightforward to compare the efficiency of one method to another, under a range of conditions/cutpoints without having to grapple with a whole list or table full of numbers.

If this still doesn't make enough sense, please keep asking questions. We'll keep working on it until it makes more sense. Barry, JB, and others can help with english translations.

Even if all we learn is the vocabulary, it become more evident to our detractors that polygraph professionals are aware of the correct foundational concepts and principles.

Most psychologists don't remember how most of their testing and diagnostic methods work, but they know about the general principles that make them work.

r

posted 12-14-2007 10:45 AM            

Said another way, the more area we have colored under that curve, the better our test performed.

Buster, does that make sense yet? Look at the curves and see where they are. Without knowing anything about how to do the calculations, you can see which curve is "higher" and has more area under it. You can then look at the actual figures in the table below each.

posted 12-14-2007 04:26 PM            

I will get to it and take my time until I understand it.

Edit:
OK, I have a decent grasp on it. Nelson is trying to close the gap between polygraph and science. Also, our numerical scoring system is most likely mocked by science professionals.

I see what the graph measures, with the help of Barry's translation.

Thanx for your help guys...

[This message has been edited by Buster (edited 12-14-2007).]

posted 12-15-2007 10:55 AM            

quote:

---

I have a decent grasp on it. Nelson is trying to close the gap between polygraph and science. Also, our numerical scoring system is most likely mocked by science professionals.

---

Once you see the logic of ROCs, they make good sense.

You are right about other professionals sometimes thinking our methods are a little silly. We also sometimes use some idiosyncratic language, and there are some things that we need to learn more about. But its all learnable.

If you want to see some real hemming and hawing, then just as a PPG technician to explain their data. Its quite poorly standardized. They'll say "it went up three points." And we ask "what's three points." And they can't explain, though it appears that their scaled metrics are based on scaled percentiles or McCall T transformations.

Its the same problem in polygraph. Years ago, as a therapist, I used to ask my examiner friend Lawson "what is +3," and "3 what." He's a smart guy, but I don't think he appreciated how important those simple questions are.

All of my recent study is really a result of some unlikely cross-training, in statistics, psychotherapy/psychodiagnostics and now polygraph. I've said this before; we are really just the construction crew - sawing boards and nailing things together. The really smart people are the one's who's names you'll see in the reference lists, who rarely if ever post on this board. They are the people who have done, and are still doing, the important work. There is little we have looked at that did not already exist from someone else's more important and earlier efforts.

One thing is for sure. We need a lot of people's efforts: smart scientists in research institutes, field examiners who can help develop an integrated understanding of scientific and statistical principles and field polygraph, people who can translate it all to common English, trainers who engender the right ideas and principles in new trainees, and field examiners who are willing to learn new vocabulary.

It should not be acceptable to ignore what we learn from studying data - that is what anti is for. If we reject good information, then we'll end up no better than them or their accusations.

So, thanks everyone for your patience and interest in the propeller-headed stuff. I'm off on another long snowy weekend. Now if these dagone exams didn't keep interfering with all the interesting stuff everything 'd be great.

Peace.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)